Skip to content

Build and test conda packages and wheels - #294

Merged
jharlow-intel merged 1 commit into
mainfrom
ci/intel-python-ci
Nov 20, 2026
Merged

jharlow-intel merged 1 commit into
mainfrom
ci/intel-python-ci

Conversation

@jharlow-intel

@jharlow-intel jharlow-intel commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Adds ci/ and .github/workflows/intel-ci.yml, which build the conda packages and the wheels for Linux and Windows, on every declared Python, and test each in a clean environment. The recipe skips macOS, as conda-forge/mkl_umath-feedstock does.

  • conda-recipe/ is now the only recipe: the conda-forge one that was conda-recipe-cf/, which takes its version and build number from git describe, as the recipe it replaces did. Its Linux build threads MKL with gnu_thread and llvm-openmp, as conda-forge's packages do; Windows and the wheels use intel_thread with libiomp5. conda-package.yml and conda-package-cf.yml are removed, as intel-ci.yml builds and tests the recipe.
  • The conda packages build Python 3.11 to 3.15 plus 3.14t and 3.15t, as mkl-service's do, from the python: list in conda_build_config.yaml. The wheels follow the pyproject.toml classifiers: 3.10 to 3.15 plus 3.14t and 3.15t.
  • Wheels build with cibuildwheel and icx 2026.2: on Linux from Intel's oneAPI yum repository, on Windows from conda-forge's dpcpp_win-64, which ci/install_icx.py installs. They leave MKL and the compiler runtime out. ci/build.py adds their requirements to each wheel through [tool.gh-ci.pin-compatible]: mkl from the mkl-devel version they were built with, and intel-openmp and intel-cmplr-lib-rt from the icx version, each up to its next major. The Linux before-all and ci/install_icx.py print icx's version for ci/build.py. The Linux wheels link with --disable-new-dtags, and ci/build.py writes any RUNPATH as RPATH, as meson-python's patchelf writes RUNPATH for the modules that use libmkl_umath_loops.so, so they get RPATH as the conda package does, and the wheel builds pass compile-args=-v. icx links the Linux libraries with full RELRO, which ci/build.py checks, along with a library search path inside $ORIGIN.
  • meson.build adds Library to CMake's search path on Windows, where mkl-devel puts its CMake config.
  • ci/test.py runs pytest --pyargs mkl_umath on the installed package.
  • The test jobs wait for every build, so a failed build starts no test job. A pull request builds the oldest and newest Python plus the newest free-threaded one; main and tags build them all.
  • conda-build records the source commit in each package's info/about.json.
  • ci/benchmark.py runs the asv suite in benchmarks/ against an installed package, for running the benchmarks outside intel-ci.yml, which doesn't run it: one quick sample of each benchmark for a pull request, a full measurement for a tag.

Miniforge is pinned to 26.7.2-0, as 26.9.4-0 makes setup-miniconda's conda init fail on Windows runners.

Validation: run 44584061088 on e53484d builds and tests the reduced pull request matrix on Linux and Windows. Not run yet: the full matrix, which only main and tags build.

@jharlow-intel
jharlow-intel marked this pull request as ready for review October 23, 2026 20:42
@jharlow-intel jharlow-intel self-assigned this Oct 28, 2026
@jharlow-intel
jharlow-intel force-pushed the ci/intel-python-ci branch 12 times, most recently from 02315bd to f53b1ce Compare November 3, 2026 18:00
@jharlow-intel jharlow-intel changed the title Build and test conda-recipe-cf for Intel Python CI Build and test conda packages and wheels Nov 5, 2026
@jharlow-intel
jharlow-intel force-pushed the ci/intel-python-ci branch 4 times, most recently from d7e1dc3 to 96e092f Compare November 9, 2026 21:50
Comment thread pyproject.toml Resolved
Comment thread conda-recipe/meta.yaml Resolved
Comment thread conda-recipe/build.sh Resolved
Comment thread pyproject.toml Resolved
Comment thread .github/workflows/intel-ci.yml Resolved
Comment thread conda-recipe/meta.yaml Resolved
Comment thread ci/config.yml Resolved
Comment thread AGENTS.md Resolved
Comment thread .github/workflows/intel-ci.yml Resolved
Comment thread pyproject.toml Resolved
Comment thread pyproject.toml Resolved
Comment thread ci/README.md Resolved
@jharlow-intel
jharlow-intel force-pushed the ci/intel-python-ci branch 2 times, most recently from d3ff9f1 to e53484d Compare November 19, 2026 18:05
Comment thread .github/workflows/intel-ci.yml Resolved
Comment thread AGENTS.md Resolved
Adds a ci/ directory and .github/workflows/intel-ci.yml. The workflow builds
conda-recipe/ with conda-build 26.9.1 and the wheels with cibuildwheel, for
every platform and Python version the recipe and pyproject.toml declare, and
tests each package in a clean environment with ci/test.py.

conda-recipe/ is now the only recipe, the conda-forge one that was
conda-recipe-cf/; the Intel-channel recipe, conda-package.yml, which copied
wheels out through WHEELS_OUTPUT_FOLDER, and conda-package-cf.yml are
removed, as intel-ci.yml builds and tests the same recipe; its test jobs
wait for every build, so a failed build starts none. conda_build_config.yaml
now lists the Python versions, 3.11 to 3.15 plus 3.14t and 3.15t, as
mkl-service's does. The recipe takes its version and build number from git
describe, as the recipe it replaces did. Its Linux build threads MKL with
gnu_thread and llvm-openmp, as conda-forge's packages do; Windows and the
wheels use intel_thread.

The wheels link MKL's layers, the Intel OpenMP runtime and icx's runtime.
[build-system] requires mkl-devel, and [tool.gh-ci.pin-compatible] maps mkl
to it and intel-openmp and intel-cmplr-lib-rt to icx: ci/build.py adds
Requires-Dist >= the version the build installed, < its next major, the way
conda's run_exports pin it, and fails a build that logs no version. It reads
mkl-devel's from python -m build's list, and icx's from the "Installed SDK
versions:" list the Linux before-all and ci/install_icx.py print; icx is
2026.2 on both. In [project] the runtime wheels would fail the recipe's pip
check, as conda-forge's packages have no pip metadata. ci/build.py takes
METADATA's line ending from the headers. The repair commands leave MKL's and
the compiler runtime's libraries out, and the Linux wheels are tagged
manylinux_2_28 only, like the mkl wheel.

icx already links the Linux libraries with full RELRO and a stack protector,
so the wheels need no hardening flags; LDFLAGS adds --disable-new-dtags, and
ci/build.py writes RUNPATH as RPATH, which meson-python's patchelf writes
for the modules it points at libmkl_umath_loops.so, so they get RPATH as the
conda package does, and the wheel builds pass compile-args=-v. The libintlc
exclude matches any soname. ci/build.py refuses a wheel whose ELF files lack
full RELRO. It keeps an RPATH or RUNPATH entry only when it is $ORIGIN or a
directory under it, as meson-python keeps the isolated build env's absolute
lib directory, and refuses a file left with none. It rewrites the wheels in
scratch, so a refused wheel never reaches --output, and refuses two built
packages with the same file name, which would overwrite each other there.

conda-build gets --extra-meta remote_url=... sha=... when GitHub Actions
sets GITHUB_SERVER_URL, GITHUB_REPOSITORY and GITHUB_SHA, so each conda
package records the commit it was built from in info/about.json.

ci/hooks.py tests each wheel in a venv made from conda-forge's Python, as pip
users install it. Every python: entry in conda_build_config.yaml names its
ABI, which ci/matrix.py requires. ci/matrix.py reads x86 as any x86 CPU and
osx64 as undefined, as conda-build does, and refuses a conda-build skip that
uses osx64. A pull request also builds and tests the newest free-threaded
version, and every push and tag run gets a concurrency group of its own.

@vlad-perevezentsev vlad-perevezentsev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No more comments from my side
LGTM
Thanks @jharlow-intel for all the work on this!

@jharlow-intel
jharlow-intel merged commit cdfa9ac into main Nov 20, 2026
55 checks passed
@jharlow-intel
jharlow-intel deleted the ci/intel-python-ci branch November 20, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants